This invention relates generally to file system conversion, and more particularly to a file system upgrade in a data integrity storage system.
Primary storage systems are generally optimized for performance and system availability, not for data invulnerability; they make design tradeoffs that favor speed over data recoverability. Even most backup appliances that are intended for data backup and archiving inherit the design philosophies of their primary storage predecessors. They are typically built using similar commodity disks and other components, and are not designed robustly to support long term data backup and data invulnerability. Most backup systems tend to be optimized for performance rather than data invulnerability. They assume that data is stored correctly to begin with. They do not have file systems that perform careful data writes, such as, for example, reading back data written to disk to verify the data was written correctly as this would impact speed and performance.
Storage systems such as used for backup and archive typically have a file system that organizes a plurality of disks into a logical disk groups from which RAID arrays are created. The physical disks groups are contained in enclosures referred to as shelves. These systems generally concatenate disk groups into a storage layer comprising a logical, contiguous, linear address space partitioned into one set of fixed sized blocks as a blockset over which the file system is formatted. Such a file system is referred to herein as a single virtual storage device file system because the file system abstracts multiple physical disks into one virtual storage device. An ordered list of disk groups, their sizes and their unique offsets in the linear address space is maintained, and the file system allocates fixed-size data blocks to users to store user data and metadata. Each disk and disk group has an embedded offset that is associated with its location in the address space. Accordingly, removal of any of the fixed-size blocks of the linear address space would result in “holes” (missing blocks) in the file system address space. Likewise, removal of a shelf would also result in holes. Neither the file system nor the RAID controllers can handle such holes, and the system cannot function with them. Thus, it is very difficult to remove or replace a shelf. There is no straightforward way of migrating user data and metadata off of a shelf and onto a new one without replicating all data to a replica system, destroying and re-creating the file system without the shelf that is removed, and replicating all of the data back onto the remaining disks. This is a manual, slow and expensive process, and one that risks data loss and corruption. Accordingly, it is difficult to upgrade such backup systems.
It is desirable to provide systems and methods that address the foregoing problems of upgrading data backup and recovery systems, while providing for data invulnerability and recoverability. It is to these ends that the present invention is directed.